Interface-based modeling and design of three dimensional spaces using two dimensional representations

ABSTRACT

Interface-based modeling and design of three dimensional spaces using two dimensional representations are provided herein. An example method includes converting a three dimensional space into a two dimensional space using a map projection schema, where the two dimensional space is bounded by ergonomic limits of a human, and the two dimensional space is provided as an ergonomic user interface, receiving an anchor position within the ergonomic user interface that defines a placement of an asset relative to the three dimensional space when the two dimensional space is re-converted back to a three dimensional space, and re-converting the two dimensional space back into the three dimensional space for display along with the asset, within an optical display system.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. non-provisional applicationSer. No. 17/009,662, filed at the United States Patent & Trade MarkOffice on Sep. 1 2020, which is a continuation of U.S. non-provisionalapplication Ser. No. 16/596,648, filed at the United States Patent &Trade Mark Office on Oct. 8 2019, which is a continuation of U.S.non-provisional application Ser. No. 15/404,172, filed at the UnitedStates Patent & Trade Mark Office on Jan. 11 2017, each of which ishereby incorporated herein by reference in its entirety to providecontinuity of disclosure.

FIELD OF THE INVENTION

The present technology is directed to three dimensional modeling anddesign, and more specifically, but not by limitation, systems andmethods that allow for the creation of three dimensional spacesdisplayed within optical display systems (such as a head mounteddisplay) using two dimensional renderings of the three dimensionalspaces where assets can be positioned within two dimensional renderingsand converted into three dimensional spaces that include the assets.

SUMMARY

According to some embodiments, the present technology is directed to amethod that may comprise: (a) converting a three dimensional space intoa two dimensional space using a map projection schema, wherein the twodimensional space is bounded by ergonomic limits of a human, wherein thetwo dimensional space is provided as an ergonomic user interface; (b)receiving an anchor position within the ergonomic user interface thatdefines a placement of an asset relative to the three dimensional spacewhen the two dimensional space is re-converted back to a threedimensional space; and (c) re-converting the two dimensional space backinto the three dimensional space for display along with the asset,within an optical display system.

In some embodiments, the present disclosure is directed to a system ofone or more computers which can be configured to perform particularoperations or actions by virtue of having software, firmware, hardware,or a combination of these installed on the system that in operationcauses or cause the system to perform the actions and/or method stepsdescribed herein. One or more computer programs can be configured toperform particular operations or actions by virtue of includinginstructions that, when executed by the data processing apparatus, causethe apparatus to perform the actions. One general aspect includesactions such as (a) converting a three dimensional space into a twodimensional space using a map projection schema, wherein the twodimensional space is bounded by ergonomic limits of a human; (b)receiving an anchor position within the ergonomic user interface thatdefines a placement of an asset relative to the three dimensional spacewhen the two dimensional space is re-converted back to a threedimensional space; and (c) re-converting the two dimensional space backinto the three dimensional space for display along with the twodimensional object, within an optical display system.

In another embodiment, the present disclosure is directed to a methodthat may comprise: (a) selecting coordinates for placement of an assetwithin a three dimensional rendering of a two dimensional space, whereinthe two dimensional space is bounded by ergonomic limits of a human anda model of a three dimensional display generated by an optical displaysystem; (b) generating the three dimensional rendering for displaywithin the optical display system using the ergonomic limits and themodel; and (c) displaying the asset in combination with a threedimensional rendering at the selected coordinates in such a way that theasset is displayed to the viewer using the optical display system in twodimensions while the three dimensional rendering is displayed to theviewer as a three dimensional image.

In another embodiment, the present disclosure is directed to a methodthat may comprise: (a) defining three dimensional positioning of virtualcameras that represent eyes of a viewer, and an inner-pupillarydistance; (b) establishing a field of view for an optical displaysystem; (c) selecting a map projection schema to convert a threedimensional space into a two dimensional space; (d) determining a stereoview frustum of the virtual cameras; (e) defining a desired distance forplacement of elements in the three dimensional space; (f) establishing atwo dimensional reference plane that intersects the stereo view frustumat the desired distance; (g) defining ergonomic limits of a human by:(i) rotating the virtual model in various directions using maximumcomfortable head movements; and (ii) defining two dimensional labels inthe three dimensional space that represent the head movements; and (h)generating an ergonomic user interface based on the ergonomic limits ofthe human using the map projection schema to create the two dimensionalspace.

The systems and methods disclosed herein can be embodied in a means forarrangement and/or steps for configurations in some embodiments.Additionally, the methods described herein can be embodied on anon-transitory computer readable medium that can be executed by acomputing device.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed disclosure, and explainvarious principles and advantages of those embodiments.

The methods and systems disclosed herein have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present disclosure so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

FIG. 1A illustrates an example three dimensional viewable space, whichcould include an image.

FIG. 1B illustrates a two dimensional representation of the threedimensional viewable space of FIG. 1A with indicia that represents humanergonomic viewing limits.

FIG. 2A illustrates an example map projection schema that can be used toconvert the three dimensional viewable space into a two dimensionalspace.

FIG. 2B illustrates the application of the example map projection schemato convert the three dimensional viewable space into a two dimensionalspace.

FIG. 3 is a schematic diagram that illustrates modeling of virtualcameras that represent eyes of a viewer, and an inner-pupillarydistance, where the lines are indicative of view frustums of the virtualcameras.

FIG. 4 is a schematic diagram that illustrates a reference plane or lineused to delineate a “comfort distance” from the virtual cameras (e.g.,modeled eyes).

FIG. 5 is a schematic diagram that illustrates an example stereo viewfrustum created from the modeling of the virtual cameras.

FIG. 6 is a schematic diagram that illustrates a reference plane set ata desired distance.

FIG. 7 is a schematic diagram that illustrates views taken at multiplehead positions using ergonomic limits of human head and/or eye movement.

FIG. 8 illustrates an equirectangular ergonomics GUI with an asset of aGUI element associated therewith.

FIG. 9 is a screenshot of the asset in FIG. 8.

FIG. 10 is a perspective view of a three dimensional display generatedfrom the ergonomic GUI with the asset.

FIG. 11 is a close-up view illustrating the GUI element placed in threedimensions at the same location illustrated in the ergonomics GUI.

FIG. 12 is yet another view similar to FIG. 10 with the total sphericalspace beyond the ergonomic limits illustrated.

FIG. 13 is an alternative embodiment of an equirectangular ergonomicsGUI with an additional mask that can be overlaid which mocks up a user'sfield of view.

FIG. 14 is a close of up view of the masked portion of theequirectangular ergonomics GUI of FIG. 13.

FIG. 15 is another view of the equirectangular ergonomics GUI of FIG. 13where the mask has been moved downward relative to FIG. 13.

FIG. 16 is a close of up view of the masked portion of theequirectangular ergonomics GUI of FIG. 15.

FIGS. 17A and 17B collectively illustrate a visual acuity heat map andits application to an ergonomics GUI.

FIG. 18 is a flowchart of an example method for modeling a human headand eyes, as well as a three dimensional environment for display.

FIG. 19 is a flowchart of an example method for determining ergonomiclimits of a human that are used to constrain an ergonomics GUI.

FIG. 20 is a method for converting a three dimensional viewable spaceinto an ergonomics GUI, as well as receiving of an asset within theergonomics GUI and reconversion of the ergonomics GUI back into a threedimensional viewable space.

FIG. 21 is an example computing device that can be used to practiceaspects of the present technology.

DETAILED DESCRIPTION

The present disclosure is generally related to three dimensionalmodeling of content for display on an optical display system such as ahead mounted display (HMD), although other types of display system suchas helmet mounted displays (e.g., virtual reality headsets) includingboth monocular and binocular helmet mounted displays, head-up displays(HUD), as well as other similar devices and systems that project and/ordisplay two dimensional and three dimensional space and objects.

Generally, the methods and systems herein can be used to model a humanhead and eyes to determine human ergonomic limits. A three dimensionalviewable space produced by the optical display system is also modeled.To be sure, the attributes of the three dimensional viewable space mayvary according to the type of optical display system being modeled.Thus, while the human ergonomic limits for a single prototypical usermay be uniform, the three dimensional viewable space generated bydifferent optical display system are often different in shape and size.For example, some optical display systems use a semi-spherical threedimensional viewable space, while others project a nearly sphericalthree dimensional viewable space. Thus, modeling of the threedimensional viewable space allows for the accommodation of many types ofoptical display systems.

The methods and systems use the ergonomic limits and the modeling of thethree dimensional viewable space to create an ergonomic graphical userinterface (GUI) that can be displayed to a designer. The ergonomic GUIis a two dimensional space or representation of the modeling of thethree dimensional viewable space, in some embodiments. For example, amap projection schema, such as equirectangular projection, is used toconvert the three dimensional viewable space into a two dimensionalrepresentation (e.g., a two dimensional image). To be sure, the presenttechnology is not limited to the use of equirectangular projection andany method for converting a three dimensional space into a twodimensional space that would be known to one of ordinary skill in theart with the present disclosure before them is likewise contemplated foruse.

The ergonomic GUI is bounded by the human ergonomic limits mentionedabove. For example, the ergonomic GUI includes the two dimensionalrendering with indicia that reference the various viewing angles for thehuman. This information is advantageous for allowing designers to placeassets in the three dimensional viewable space in the line of sight ofthe viewer (or in proximity thereof) for optimal viewing.

The ergonomic GUI can be used by a designer to quickly and efficientlyaugment the three dimensional viewable space by placement/anchoring ofassets into its two dimensional representation in the ergonomic GUI. Thesystems and methods allow for toggling between the two dimensional spaceand the three dimensional space to visually verify optimal placement ofassets within the three dimensional viewable space.

For context, the present technology solves a problem for a developer ordesigner of augmented reality and/or virtual reality content. Namely,the developer desires a way of organizing three dimensional and/or twodimensional content around a viewer, which may involve placing content(e.g., assets) around the viewer in several places in a threedimensional viewable space. Yet designing much of this three dimensionalviewable space (3D GUI) happens in software using two dimensionalrepresentations, and as a developer it is often unclear where exactlyalong a sphere a two dimensional or three dimensional asset shouldexist. Ergonomic factors or limits of a human being's ability to see thecontent are also a consideration.

The present technology provides a solution, which comprises representinga three dimensional viewable space in a two dimensional manner, wherethe two dimensional space is optimized for designing and spatiallyplacing assets that also addresses the ergonomic limits of head rotationand natural lines of sight. This two dimensional space/representationcould be used to both design the size and look of assets, and representtheir locations around the user. The two dimensional representationcould be automatically re-projected into the three dimensional viewablespace (e.g., toggled), eliminating or reducing the lengthy and complexprocess of positioning these assets in the three dimensional viewablespace manually.

In some embodiments, the designer can employ masking or display avisual-acuity-heat-map within the ergonomic GUI to further improveoptimal placement of assets for eventual display when the threedimensional viewable space is rendered on a head mounted display.

These and other advantages of the present disclosure are described ingreater detail throughout with reference to the collective drawings(FIGS. 1A-21).

Referring now to FIGS. 1A and 1B collectively, a three dimensionalviewable space 100 is illustrated. The three dimensional viewable space100, in this embodiment, comprises a semi-spherical shaped space that isgenerated by an example optical display system. Again, the opticaldisplay system can include any augmented reality or virtual realityproducing device. The three dimensional viewable space 100 can comprisean image that is converted into three dimensional viewing by any methodor process that would be known to one of ordinary skill in the art withthe present disclosure before them. While a spherical shape iscontemplated, any number of shapes such as cubic, cylindrical, or otherpolygonal three dimensional shapes can also be utilized.

The three dimensional viewable space 100 is then converted into a twodimensional space/representation 102 through use of a map projectionschema. In one embodiment, the map projection schema comprisesequirectangular projection. Other map projection schemas include but arenot limited to Cassini, Mercator, Cubemap, Web Mercator, Gauss-Kruger,and Gall stereographic—just to name a few.

In some embodiments, the two dimensional space 102 is a two dimensionalrendering of the three dimensional viewable space 100 with indiciadisplayed thereon. The indicia demarcate human ergonomic limits andinclude any of limits of human eye movement from side-to-side,up-to-down, and eye rotation, as well as head movement from side-to-sideand up-to-down. These human ergonomic limits refer to the maximum limitsof these physical movements so as to avoid pain and/or injury. While thesystems and methods herein can utilize basic human ergonomic limits as abasis for determining the indicia placed on the two dimensional space102, the human ergonomic limits can be determined on an individualbasis. Thus, individuals with limited range of motion can beaccommodated such that three dimensional viewable spaces are createdthat are optimal for the individual.

Thus, prior to displaying a three dimensional viewable space to aviewer, a virtual model is created for a viewer's head, eyes, and insome embodiments a particular optical display system.

FIG. 2A illustrates an example equirectangular projection mapping schema200. FIG. 2B illustrates the equirectangular projection mapping schema200 as applied to the converted two dimensional space version 102 of thethree dimensional space of FIG. 1A.

In one embodiment, a server is configured to provide the modeling,rendering, and provision of ergonomic GUIs for a developer. An examplecomputing system is described with reference to FIG. 21. The server inthis instance comprises a specific-purpose computing device that isprogramed to perform various operations that are not well-known,ordinary, well-understood, or generic computing functions.

Referring now to FIG. 3, in one or more embodiments, the server isconfigured to create a virtual model of a user's head, eyes, and anoptical display of an optical display system. Some embodiments includedetermining three dimensional positions of virtual cameras thatrepresent the human's eyes, their inner-pupillary distance, and anyother parameter as required by the specific optical display system forwhich the renderings are designed for display.

A human head 300 is represented, which comprises two eyes 302 and 304. Aposition and placement of virtual cameras 306 and 308 are determined forthe two eyes 302 and 304, respectively. Lines 310A-F represent viewfrustums of each OpenGI cameras (e.g., virtual human eyes), any threedimensional object in these pyramid shapes are drawn to the displays. Anear clipping plane is aligned with the active area of the opticaldisplay system (for example a Lumus™ display). Lines 312A and 3128 arenear-frustum. Objects within in this area are not drawn to the displaysin some embodiments.

In FIG. 4, line 314 represents a “comfort distance” relative to aviewer's face. In one embodiment, the comfort distance equals sixinches, although other distances can be used depending on thepreferences of the viewer. Content placed any closer than the comfortdistance would be uncomfortable to view, but again this distance isvariable.

The generation of view frustums and imposition of a comfort distanceplane is generally referred to as the determination of a field of viewfor the optical display system and the resulting OpenGL projectionmatrix used to draw virtual elements to the screen or display of theoptical display system.

Next, the server performs operations comprising determining a “stereoview frustum” (the three dimensional space in which virtual contentwould be seen by both eyes) which is the additive shape of the left eyeand right eye view frustum. This stereo view frustum 316 is illustratedin FIG. 5. To be sure, the present disclosure can be adapted for usewith mono view frustum technology as well as stereo views.

FIG. 5 illustrates an example stereo view frustum 316 which represents athree dimensional space in which virtual content would be seen by botheyes, which comprises an additive shape of the left eye view frustum andthe right eye view frustum. Virtual content within area 318 is viewableby both virtual cameras and is thus drawn on both virtual displays, andviewable in stereo three dimensions by human eyes.

The server then allows for specification of a desired distance thatvirtual elements should exist from a viewer, which is dependent on afocal length of the optical display system. For multiple distances, thefollowing steps can be repeated for each distance (processes associatedwith the descriptions of FIGS. 6 and 7).

As illustrated in FIG. 6, in some embodiments, the server draws a twodimensional plane 320 (illustrated in an expanded view as a projection321) in three dimensional space, intersecting the stereo view frustum316 at a desired distance from the virtual cameras 306 and 308. This twodimensional plane 321 represents a single field of view of the viewer.The two dimensional plane 320, placed at two meters from a Lumus™display (an example optical display system) was rendered and labels wereadded that describe ergonomic information. By way of non-limitingexample, ergonomic information was obtained from “The Measurement ofMan” by Henry Dreyfuss and combined with limits of the modeled displays.

The server can then duplicate the entire model repeating theabove-mentioned steps for multiple head rotations. For example, thevirtual model of the head is then rotated to either side by a maximumcomfortable head rotation according to ergonomic limits of averagehumans, or even for specific users. FIG. 7 illustrates a plurality ofpositions taken at 45 degree intervals (e.g., a maximum comfortable headrotation according to Dreyfuss). The distance that content should appearfrom the user is represented by lines 324A-C, in some embodiments thisis the focal length of the head mounted display.

The server can then rotate the virtual head(s) several more times torepresent comfortable head rotation ranges and maximum head rotations.The limits are labeled by the server in three dimensional space and thisinformation is projected into a two dimensional representation using anequirectangular projection matrix, as illustrated in FIGS. 2A and 2B.

The processes above can also be used to generate two dimensional labels,which are referred to as indicia, in the three dimensional space forestablishing a vertical midline as a maximum comfortable head rotationleftward.

The above process can also be repeated as many times as desired to coverthe total ergonomic limits of average viewers, or a specific user.

To account for and remedy distortion artifacts, the server can re-traceelements of the two dimensional equirectangular projection. For exampletaking the stereo view frustum and redrawing an upper-left point of theplane to an upper right point, effectively removing a distorted line andreplacing it with a straight line.

The two dimensional representation with indicia is referred to as anergonomic GUI.

In one embodiment, the server can specify anchor points that determinethe placement of an asset added by the developer. In some embodiment theserver can use a midpoint position of the asset as a representation ofwhere, in three dimensional space, the asset should be located aroundthe viewer (at the defined depth determined infra). In some embodiments,the server does not render everything designed and literally project theimages and assets in three dimensions, as doing so would create a lot ofundesired distortion artifacts. The server can overlay the asset,whether it is two dimensional, two and a half dimensional, or threedimensional, at the anchor point, while projecting the three dimensionalspace in three dimensions without distortion. Thus, in some embodiments,a two dimensional asset will appear to the viewer in two dimensionswithin the display while the remainder of the display is viewable inthree dimensions.

FIG. 8 illustrates an example two dimensional space/representation thatcomprises, for example, the two dimensional space 102 of FIG. 1B whichhas been augmented with indicia to create an ergonomic GUI 400.

In general, the ergonomic GUI 400 is a design and engineering suite thatdescribes a three dimensional space around a user at a specifieddistance (usually the focal length of the HMD) in the form of a twodimensional document (e.g., two dimensional space/representation)convenient for designing. The ergonomic GUI 400 is useful for describingwhere applications and UI content should exist around a viewer in threedimensional spaces.

The ergonomic GUI 400 is a representation of a portion of a sphericalspace (or other similar three dimensional shape) in some embodiments.For example, the ergonomic GUI 400 illustrates approximately 120 degreesof a total three dimensional space that has been mapped from threedimensions to two dimensions using an equirectangular projection.

The ergonomic GUI 400 also incorporates indicia that represent ergonomiclimits of head rotation and natural lines of sight, in some embodiments.The ergonomic GUI 400 is useful for planning out where applicationsexist around a user and predicting which areas are uncomfortable toview. For example, the indicia can represent maximum comfortable headrotation angles. The region within those lines are referred to as aninteraction zone, which is a viewing space where a viewer cancomfortably view and/or dwell on assets a given area.

The indicia can include ergonomic limits determined during the modelingprocess described infra. For example, the indicia comprises matrix linesthat create a grid that divide the two dimensional space 102 intosegments. The longitudinal lines, such as line 402 represent headrotation angles along the yaw axis (e.g., left-to-right). Latitudinallines, such as line 404 represent head rotation angles along the pitchaxis (e.g., up and down). Longitudinal lines 406 and 408 illustrate leftand right maximum head rotation limits, respectively. Latitudinal lines410 and 412 illustrate up and down maximum head pivot limits,respectively. To be sure, while these ergonomic limits reference headrotation and pivot, the ergonomic limits can equally be used torepresent eye movement limitations only and/or combinations of both headrotation and pivot limits and eye movement limits. Additionally, eyerotations can also be considered and mapped as part of the indicia.

For example, line 414 represents a line of sight of the viewer and line416 represents a maximum downward eyeball rotation that can be performedby a viewer before having to pivot their head downward to see something.Thus, if the developer desires to place an asset for optimal viewing bya viewer so as not to require the user to have to pivot their headdownwardly, the developer can place the asset at least above line 416.In one embodiment, a developer places an asset 422, which includes a GUImenu above the maximum downward eyeball rotation line 416, near thecenter of the viewer's line of sight.

In some embodiments, the ergonomic GUI 400 comprises absolute visuallimits that can also be illustrated using lines 418 and 420.

A perspective view of the asset 422 is illustrated in FIG. 9 forclarity.

After the developer has placed the asset 422 into the ergonomic GUI 400,the server determines a midpoint of the asset 422 and creates an anchorpoint 424 that is aligned with the midpoint of the asset 422. The serverstores the position of the anchor point 424 prior to converting theergonomic GUI 400 and the two dimensional space 102 back into a threedimensional space/rendering.

This anchor point 424 allows the server to re-render the threedimensional space and then place the asset 422 into the threedimensional space at the anchor point 424. Thus, in some embodiments,the asset 422 is placed into or overlaid onto the three dimensionalspace such that the asset 422 is not directly projected into the threedimensional space which would potentially lead to distortion of theasset 422 when rendered. The asset 422 can float in two or threedimensional space.

FIG. 10 illustrates the three dimensional space 500, which is are-rendered version of the two dimensional space 102 (also see FIG. 1Bor FIG. 8) that was augmented with the asset 422. Indeed, the threedimensional space 500 comprises the two dimensional space rendered backinto the three dimensional space 102 of FIG. 1A, with the addition ofthe asset 422.

The three dimensional space 500 is a spherical version of the ergonomicGUI 400 of FIGS. 8 and 9, for illustrative purposes so as to clarifythat the placement of the asset 422 is in accordance with its placementin the ergonomic GUI 400.

FIG. 11 is a close up view of FIG. 10 and illustrates the asset 422 asbeing placed into the three dimensional space 500 as an overlay, suchthat the asset 422 does not take the shape of the three dimensionalspace 500, which in this instance is a curvature. The asset 422 appearsas a two dimensional flat GUI menu that appears to float over the threedimensional space 500.

FIG. 12 is a further expanded view of FIG. 10.

According to some embodiments, the present disclosure provides varioustools that allow a designer or developer to strategically place assetswithin an ergonomic GUI. Example tools mimic a field of view of anexample viewer or model the field of view using a heat map.

FIGS. 13 and 14 collectively illustrate an example FOV (field of view)mask 600 that is applied to an example ergonomic GUI 602. The exampleFOV mask 600 can be overlaid on the ergonomic GUI 602 and mocks up aviewer's FOV. That is, a space within the masked area is what a viewercould see at once without moving their eyes or head.

Only a portion of an asset 604 is visible within the space of the FOVmask 600. Thus, if the developer desires for the asset 604 to be visiblewithout requiring the viewer to move their head or eyes, the developercan move the asset higher into the area defined by the FOV mask 600.

FIGS. 15 and 16 collectively illustrate movement of the FOV mask 600downwardly. The horizontal midline H is now at −10 degrees vertical,which means the viewer would have to rotate their head 10 degrees towardthe floor in order to see the space within the FOV mask 600 and view thewhole GUI element. This FOV modeling is useful because it means thatbefore any code is written in the development process, a GUI designercan simulate a viewer's interaction with display generated by an opticaldisplay system.

FIGS. 17A and 17B collectively illustrate a visual acuity heat-map 700.The visual acuity heat-map 700 comprises a pixelated or block-coloredrepresentation of a viewer's acuity within a specified field of view.Cooler hues near the middle and upper middle of the visual acuityheat-map 700 indicate higher acuity than areas around the periphery ofthe visual acuity heat-map 700 and towards the lower edge.

The visual acuity heat-map 700 can be applied to an example ergonomicGUI 702. In some embodiments, the latitudinal and longitudinal linesthat represent the ergonomic limits of the viewer form segments, such assegment 704. Each segment comprises a portion of a three dimensionalrepresentation that will be viewable by the viewer at one time withoutmoving their head and/or eyes. The visual acuity heat-map 700, whenapplied to a segment of the ergonomic GUI 702, indicates optimal viewingportions of the segment into which assets can be placed for optimalviewing based on the acuity of the viewer.

Referring now to FIG. 18, which illustrates an example method of thepresent disclosure. The method comprises a step 800 of defining threedimensional positioning of virtual cameras that represent eyes of aviewer, and an inner-pupillary distance. Next, the method includes astep 802 of establishing a field of view for an optical display system.Again, this can include any optical display system such as an augmentedreality or virtual reality device.

In some embodiments, the method comprises a step 804 of selecting a mapprojection schema to convert a three dimensional space into a twodimensional space and a step 806 of determining a stereo view frustumfor the virtual cameras.

In one embodiment, the method comprises a step 806 of defining a desireddistance for placement of elements in the three dimensional space, whichis based on operational parameters of the optical display system.

In some embodiments, the method includes a step 808 of establishing atwo dimensional reference plane that intersects the stereo view frustumat the desired distance, which sets the desired distance referenced instep 806.

The steps of the flowchart of the method of FIG. 18 provide a means formodeling an optical display system and a viewer's head and eyes relativeto the optical display system.

FIG. 19 illustrates another example method for defining ergonomic limitsof a human (e.g., viewer). The method begins with a step 900 of rotatingthe virtual model generated in the method of FIG. 18 in variousdirections using maximum comfortable head movements.

At each location, the method includes a step 902 of defining twodimensional labels in the three dimensional space that are based on thehead movements. For example, maximum left and right head rotationpositions and maximum up and down head pivot positions are labeled.Again, these head movements can be tailored to the specific limitationsof the viewer, and even individuals with physical limitations.

Next, the method comprises a step 904 of generating an ergonomic userinterface based on the ergonomic limits of the human using the mapprojection schema to create the two dimensional space.

FIG. 20 illustrates another method, which includes the creation of a twodimensional space from a three dimensional space and reconversion of thetwo dimensional space back into the three dimensional space afterreceiving assets.

In one embodiment, the method includes a step 1000 of converting a threedimensional space into a two dimensional space using a map projectionschema. As mentioned above, any map projection schema can be utilized.In some embodiments, the two dimensional space is bounded by ergonomiclimits of a human which were determine in the methods of FIGS. 18 and19, for example.

Next, the method comprises a step 1002 of receiving an anchor positionwithin the ergonomic user interface that defines a placement of an assetrelative to the three dimensional space when the two dimensional spaceis re-converted back to a three dimensional space. For example, the twodimensional space is used to generate the ergonomic user interface thatincludes a two dimensional image created using the map projectionschema. Ergonomic limits are defined on the two dimensional image asvarious lines that indicate the ergonomic limits.

A developer uses the ergonomic user interface to place the asset forpositioning. Once placed, the asset is anchored and then the methodincludes a step 1004 of re-converting the two dimensional space backinto the three dimensional space for display along with the asset,within an optical display system.

FIG. 21 is a diagrammatic representation of an embodiment of a machinein the form of a computer system 1, within which a set of instructionsfor causing the machine to perform any one or more of the methodologiesdiscussed herein may be executed. In various example embodiments, themachine operates as a standalone device or may be connected (e.g.,networked) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client machine in aserver-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine may be arobotic construction marking device, a base station, a personal computer(PC), a tablet PC, a set-top box (STB), a personal digital assistant(PDA), a cellular telephone, a portable music player (e.g., a portablehard drive audio device such as an Moving Picture Experts Group AudioLayer 3 (MP3) player), a web appliance, a network router, switch orbridge, or any machine capable of executing a set of instructions(sequential or otherwise) that specify actions to be taken by thatmachine. Further, while only a single machine is illustrated, the term“machine” shall also be taken to include any collection of machines thatindividually or jointly execute a set (or multiple sets) of instructionsto perform any one or more of the methodologies discussed herein.

The embodiment of the computer system 1 includes a processor or multipleprocessors 5 (e.g., a central processing unit (CPU), a graphicsprocessing unit (GPU), or both), and a main memory 10 and static memory15, which communicate with each other via a bus 20. The computer system1 may further include a video display 35 (e.g., a liquid crystal display(LCD)). The computer system 1 may also include an alpha-numeric inputdevice(s) 30 (e.g., a keyboard), a cursor control device (e.g., amouse), a voice recognition or biometric verification unit (not shown),a drive unit 37 (also referred to as disk drive unit), a signalgeneration device 40 (e.g., a speaker), and a network interface device45. The computer system 1 may further include a data encryption module(not shown) to encrypt data.

The drive unit 37 includes a computer or machine-readable medium 50 onwhich is stored one or more sets of instructions and data structures(e.g., instructions 55) embodying or utilizing any one or more of themethodologies or functions described herein. The instructions 55 mayalso reside, completely or at least partially, within the main memory 10and/or within the processors 5 during execution thereof by the computersystem 1. The main memory 10 and the processors 5 may also constitutemachine-readable media.

The instructions 55 may further be transmitted or received over anetwork via the network interface device 45 utilizing any one of anumber of well-known transfer protocols (e.g., Hyper Text TransferProtocol (HTTP)). While the machine-readable medium 50 is shown in anexample embodiment to be a single medium, the term “computer-readablemedium” should be taken to include a single medium or multiple media(e.g., a centralized or distributed database and/or associated cachesand servers) that store the one or more sets of instructions. The term“computer-readable medium” shall also be taken to include any mediumthat is capable of storing, encoding, or carrying a set of instructionsfor execution by the machine and that causes the machine to perform anyone or more of the methodologies of the present application, or that iscapable of storing, encoding, or carrying data structures utilized by orassociated with such a set of instructions. The term “computer-readablemedium” shall accordingly be taken to include, but not be limited to,solid-state memories, optical and magnetic media, and carrier wavesignals. Such media may also include, without limitation, hard disks,floppy disks, flash memory cards, digital video disks, random accessmemory (RAM), read only memory (ROM), and the like. The exampleembodiments described herein may be implemented in an operatingenvironment comprising software installed on a computer, in hardware, orin a combination of software and hardware.

Not all components of the computer system 1 are required and thusportions of the computer system 1 can be removed if not needed, such asInput/Output (I/O) devices (e.g., input device(s) 30). One skilled inthe art will recognize that the Internet service may be configured toprovide Internet access to one or more computing devices that arecoupled to the Internet service, and that the computing devices mayinclude one or more processors, buses, memory devices, display devices,input/output devices, and the like. Furthermore, those skilled in theart may appreciate that the Internet service may be coupled to one ormore databases, repositories, servers, and the like, which may beutilized in order to implement any of the embodiments of the disclosureas described herein.

As used herein, the term “module” may also refer to any of anapplication-specific integrated circuit (“ASIC”), an electronic circuit,a processor (shared, dedicated, or group) that executes one or moresoftware or firmware programs, a combinational logic circuit, and/orother suitable components that provide the described functionality.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present technology has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the present technology in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the presenttechnology. Exemplary embodiments were chosen and described in order tobest explain the principles of the present technology and its practicalapplication, and to enable others of ordinary skill in the art tounderstand the present technology for various embodiments with variousmodifications as are suited to the particular use contemplated.

Aspects of the present technology are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thepresent technology. It will be understood that each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present technology. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

In the following description, for purposes of explanation and notlimitation, specific details are set forth, such as particularembodiments, procedures, techniques, etc. in order to provide a thoroughunderstanding of the present invention. However, it will be apparent toone skilled in the art that the present invention may be practiced inother embodiments that depart from these specific details.

Reference throughout this specification to “one embodiment” or “anembodiment” means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of the phrases “in one embodiment” or “in an embodiment” or“according to one embodiment” (or other phrases having similar import)at various places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments. Furthermore, depending on the context ofdiscussion herein, a singular term may include its plural forms and aplural term may include its singular form. Similarly, a hyphenated term(e.g., “on-demand”) may be occasionally interchangeably used with itsnon-hyphenated version (e.g., “on demand”), a capitalized entry (e.g.,“Software”) may be interchangeably used with its non-capitalized version(e.g., “software”), a plural term may be indicated with or without anapostrophe (e.g., PE's or PEs), and an italicized term (e.g., “N+1”) maybe interchangeably used with its non-italicized version (e.g., “N+1”).Such occasional interchangeable uses shall not be consideredinconsistent with each other.

Also, some embodiments may be described in terms of “means for”performing a task or set of tasks. It will be understood that a “meansfor” may be expressed herein in terms of a structure, such as aprocessor, a memory, an I/O device such as a camera, or combinationsthereof. Alternatively, the “means for” may include an algorithm that isdescriptive of a function or method step, while in yet other embodimentsthe “means for” is expressed in terms of a mathematical formula, prose,or as a flow chart or signal diagram.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

If any disclosures are incorporated herein by reference and suchincorporated disclosures conflict in part and/or in whole with thepresent disclosure, then to the extent of conflict, and/or broaderdisclosure, and/or broader definition of terms, the present disclosurecontrols. If such incorporated disclosures conflict in part and/or inwhole with one another, then to the extent of conflict, the later-dateddisclosure controls.

The terminology used herein can imply direct or indirect, full orpartial, temporary or permanent, immediate or delayed, synchronous orasynchronous, action or inaction. For example, when an element isreferred to as being “on,” “connected” or “coupled” to another element,then the element can be directly on, connected or coupled to the otherelement and/or intervening elements may be present, including indirectand/or direct variants. In contrast, when an element is referred to asbeing “directly connected” or “directly coupled” to another element,there are no intervening elements present. The description herein isillustrative and not restrictive. Many variations of the technology willbecome apparent to those of skill in the art upon review of thisdisclosure.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. The descriptions are not intended to limit the scope of theinvention to the particular forms set forth herein. To the contrary, thepresent descriptions are intended to cover such alternatives,modifications, and equivalents as may be included within the spirit andscope of the invention as defined by the appended claims and otherwiseappreciated by one of ordinary skill in the art. Thus, the breadth andscope of a preferred embodiment should not be limited by any of theabove-described exemplary embodiments.

1. A method, comprising: converting a three dimensional shape space intoa two dimensional space; generating, for display, an ergonomic userinterface comprising the two dimensional space, receiving an anchorposition within the ergonomic user interface that defines a placement ofan asset relative to the three dimensional shape space when the twodimensional space is re-converted back to a three dimensional space,re-converting the two dimensional space back into the three dimensionalshape space for display along with the asset, within an optical displaysystem; and displaying the re-converted three dimensional shape space inthe optical display system, the re-converted three dimensional shapespace comprising the asset located in a line of sight of the viewer. 2.The method according to claim 1, further comprising projecting the assetinto the optical display system at the anchor point such that the asset,wherein the asset comprises any of a two dimensional object and a threedimensional object.
 3. The method according to claim 1, furthercomprising allowing a designer to toggle between the two dimensionalspace and the three dimensional shape space; and wherein the step ofconverting a three dimensional shape space into a two dimensional spacecomprises converting the three dimensional shape space into a twodimensional space using a map projection schema that includes at leastone projection type.
 4. The method according to claim 3, wherein the mapprojection schema optimizes the two dimensional space for receiving andspatially orienting the asset.
 5. The method according to claim 3,wherein the ergonomic user interface comprises: the asset overlaid uponthe two dimensional space; and a plurality of indicia that indicateviewing angles for a viewer that are based on the ergonomic limits thatare selected from any of eye rotation, head rotation, natural lines ofsight of a viewer, and any combinations thereof.
 6. The method accordingto claim 3, further comprising generating a virtual model of a head andeyes of the viewer by: defining three dimensional positioning of virtualcameras that represent the eyes of the viewer and an inner-pupillarydistance; determining a field of view of the optical display system;selecting the map projection schema; determining a stereo view frustum;defining a desired distance for placement of three dimensional elementsin the three dimensional space as well as the asset; and establishing atwo dimensional reference plane that intersects the stereo view frustumat the desired distance.
 7. The method according to claim 6, whereinergonomic limits of head rotation and natural lines of sight of theviewer are determined by: rotating the virtual model of the head invarious directions using maximum comfortable head movements; anddefining two dimensional labels in the three dimensional shape spacethat represent: (1) leftmost maximum comfortable head rotation; (2) arightmost maximum comfortable head rotation; (3) an upward maximumcomfortable head tilt; and (4) a lower maximum comfortable head tilt. 8.A method, comprising: converting a three dimensional shape space into atwo dimensional space; selecting coordinates for placement of an assetwithin a three dimensional rendering of the two dimensional space,generating, for display, an ergonomic user interface comprising the twodimensional space, generating the three dimensional rendering fordisplay within the optical display system, wherein the three dimensionalrendering comprises the two dimensional space that has been renderedinto the three dimensional shape space for display along with the asset;and displaying the asset in combination with the three dimensionalrendering at the selected coordinates in such a way that the asset isdisplayed to the viewer using the optical display system in twodimensions while the three dimensional rendering is displayed to theviewer as a three dimensional image.
 9. The method according to claim 8,wherein the step of converting a three dimensional shape space into atwo dimensional space comprises converting the three dimensional shapespace into a two dimensional space using a map projection schema thatincludes at least one projection type.
 10. The method according to claim8, further comprising allowing a user to toggle between the twodimensional space and the three dimensional rendering.
 11. The methodaccording to claim 10, further comprising: generating a view of the twodimensional space; and receiving the asset, wherein the selection of thecoordinates is determined by placement of the asset onto the twodimensional space.
 12. The method according to claim 11, furthercomprising applying a masking frame to at least a portion of the view ofthe two dimensional space, wherein the masking frame defines a selectedportion of a focal point of the viewer.
 13. The method according toclaim 8, wherein the asset is displayed in such a way that the asset iscoplanar with a reference plane that extends through the threedimensional rendering.
 14. The method according to claim 8, wherein theasset is displayed in such a way that the asset at least partiallyconforms to a shape of the three dimensional rendering.
 15. The methodaccording to claim 8, further comprising outputting a location of theasset relative to horizontal and vertical angles of the threedimensional rendering.
 16. The method according to claim 8, furthercomprising determining if a placement of the asset is outside of avisual acuity heat-map.
 17. The method according to claim 8, furthercomprising providing an ergonomic template that is a two dimensionalrepresentation of the three dimensional shape space, which comprises athree dimensional shape space covering approximately 120 degrees ofhorizontal and vertical viewing angles, the three dimensional shapespace being mapped to a two dimensional space using a map projectionschema.
 18. The method according to claim 17, wherein the twodimensional image of the three dimensional shape space is subdividedinto a plurality of sections, further wherein each of the plurality ofsections presents a separate field of view of the two dimensional spacethat can be displayed within the optical display system.
 19. The methodaccording to claim 18, further comprising applying a visual acuityheat-map to each of the plurality of sections to determine if the assetmapped to the two dimensional space will be visible when the twodimensional space is converted to the three dimensional rendering fordisplay within the optical display system.
 20. A method for generating avirtual model, comprising: defining three dimensional positioning ofvirtual cameras that represent eyes of a viewer, and an inner-pupillarydistance; establishing a field of view for an optical display system;converting a three dimensional shape space into a two dimensional space;determining a stereo view frustum; defining a desired distance forplacement of elements in the three dimensional shape space; establishinga two dimensional reference plane that intersects the stereo viewfrustum at the desired distance; generating an ergonomic user interfaceto create a two dimensional space from the three dimensional shapespace, re-converting the two dimensional space back into the threedimensional shape space for display along with the asset, within anoptical display system; and displaying the re-converted threedimensional shape space in the optical display system, the re-convertedthree dimensional shape space comprising the asset located in a line ofsight of the viewer.